Nonparametric estimation of the density of the alternative hypothesis in a multiple testing setup. Application to local false discovery rate estimation

摘要

In a multiple testing context, we consider a semiparametric mixture modelwith two components where one component is known and corresponds to thedistribution of $p$-values under the null hypothesis and the other component$f$ is nonparametric and stands for the distribution under the alternativehypothesis. Motivated by the issue of local false discovery rate estimation, wefocus here on the estimation of the nonparametric unknown component $f$ in themixture, relying on a preliminary estimator of the unknown proportion $\theta$of true null hypotheses. We propose and study the asymptotic properties of twodifferent estimators for this unknown component. The first estimator is arandomly weighted kernel estimator. We establish an upper bound for itspointwise quadratic risk, exhibiting the classical nonparametric rate ofconvergence over a class of H\"older densities. To our knowledge, this is thefirst result establishing convergence as well as corresponding rate for theestimation of the unknown component in this nonparametric mixture. The secondestimator is a maximum smoothed likelihood estimator. It is computed through aniterative algorithm, for which we establish a descent property. In addition,these estimators are used in a multiple testing procedure in order to estimatethe local false discovery rate. Their respective performances are then comparedon synthetic data.